A shadowing problem in the detection of overlapping communities: lifting the resolution limit through a cascading procedure

Community detection is the process of assigning nodes and links in significant communities (e.g. clusters, function modules) and its development has led to a better understanding of complex networks. When applied to sizable networks, we argue that most detection algorithms correctly identify prominent communities, but fail to do so across multiple scales. As a result, a significant fraction of the network is left uncharted. We show that this problem stems from larger or denser communities overshadowing smaller or sparser ones, and that this effect accounts for most of the undetected communities and unassigned links. We propose a generic cascading approach to community detection that circumvents the problem. Using real and artificial network datasets with three widely used community detection algorithms, we show how a simple cascading procedure allows for the detection of the missing communities. This work highlights a new detection limit of community structure, and we hope that our approach can inspire better community detection algorithms.Comment: 14 pages, 12 figures + supporting information (5 pages, 6 tables, 3 figures

The Naming Game in Social Networks: Community Formation and Consensus Engineering

We study the dynamics of the Naming Game [Baronchelli et al., (2006) J. Stat. Mech.: Theory Exp. P06014] in empirical social networks. This stylized agent-based model captures essential features of agreement dynamics in a network of autonomous agents, corresponding to the development of shared classification schemes in a network of artificial agents or opinion spreading and social dynamics in social networks. Our study focuses on the impact that communities in the underlying social graphs have on the outcome of the agreement process. We find that networks with strong community structure hinder the system from reaching global agreement; the evolution of the Naming Game in these networks maintains clusters of coexisting opinions indefinitely. Further, we investigate agent-based network strategies to facilitate convergence to global consensus.Comment: The original publication is available at http://www.springerlink.com/content/70370l311m1u0ng3

Router-level community structure of the Internet Autonomous Systems

The Internet is composed of routing devices connected between them and organized into independent administrative entities: the Autonomous Systems. The existence of different types of Autonomous Systems (like large connectivity providers, Internet Service Providers or universities) together with geographical and economical constraints, turns the Internet into a complex modular and hierarchical network. This organization is reflected in many properties of the Internet topology, like its high degree of clustering and its robustness. In this work, we study the modular structure of the Internet router-level graph in order to assess to what extent the Autonomous Systems satisfy some of the known notions of community structure. We show that the modular structure of the Internet is much richer than what can be captured by the current community detection methods, which are severely affected by resolution limits and by the heterogeneity of the Autonomous Systems. Here we overcome this issue by using a multiresolution detection algorithm combined with a small sample of nodes. We also discuss recent work on community structure in the light of our results

Attack Resilience of the Evolving Scientific Collaboration Network

Stationary complex networks have been extensively studied in the last ten years. However, many natural systems are known to be continuously evolving at the local (“microscopic”) level. Understanding the response to targeted attacks of an evolving network may shed light on both how to design robust systems and finding effective attack strategies. In this paper we study empirically the response to targeted attacks of the scientific collaboration networks. First we show that scientific collaboration network is a complex system which evolves intensively at the local level – fewer than 20% of scientific collaborations last more than one year. Then, we investigate the impact of the sudden death of eminent scientists on the evolution of the collaboration networks of their former collaborators. We observe in particular that the sudden death, which is equivalent to the removal of the center of the egocentric network of the eminent scientist, does not affect the topological evolution of the residual network. Nonetheless, removal of the eminent hub node is exactly the strategy one would adopt for an effective targeted attack on a stationary network. Hence, we use this evolving collaboration network as an experimental model for attack on an evolving complex network. We find that such attacks are ineffectual, and infer that the scientific collaboration network is the trace of knowledge propagation on a larger underlying social network. The redundancy of the underlying structure in fact acts as a protection mechanism against such network attacks

Inference of hidden structures in complex physical systems by multi-scale clustering

We survey the application of a relatively new branch of statistical physics--"community detection"-- to data mining. In particular, we focus on the diagnosis of materials and automated image segmentation. Community detection describes the quest of partitioning a complex system involving many elements into optimally decoupled subsets or communities of such elements. We review a multiresolution variant which is used to ascertain structures at different spatial and temporal scales. Significant patterns are obtained by examining the correlations between different independent solvers. Similar to other combinatorial optimization problems in the NP complexity class, community detection exhibits several phases. Typically, illuminating orders are revealed by choosing parameters that lead to extremal information theory correlations.Comment: 25 pages, 16 Figures; a review of earlier work

The role of endogenous and exogenous mechanisms in the formation of R&D networks

We develop an agent-based model of strategic link formation in Research and Development (R&D)networks. Empirical evidence has shown that the growth of these networks is driven by mechanisms whichare both endogenous to the system (that is, depending on existing alliances patterns) and exogenous (that is, driven by an exploratory search for newcomer firms). Extant research to date has not investigated both mechanisms simultaneously in a comparative manner. To overcome this limitation, we develop a general modeling framework to shed light on the relative importance of these two mechanisms. We test our model against a comprehensive dataset, listing cross-country and cross-sectoral R&D alliances from 1984 to 2009. Our results show that by fitting only three macroscopic properties of the network topology, this framework is able to reproduce a number of micro-level measures, including the distributions of degree, local clustering, path length and component size, and the emergence of network clusters. Furthermore, by estimating the link probabilities towards newcomers and established firms from the data, we find that endogenous mechanisms are predominant over the exogenous ones in the network formation, thus quantifying the importance of existing structures in selecting partner firms

Community landscapes: an integrative approach to determine overlapping network module hierarchy, identify key nodes and predict network dynamics

Background: Network communities help the functional organization and evolution of complex networks. However, the development of a method, which is both fast and accurate, provides modular overlaps and partitions of a heterogeneous network, has proven to be rather difficult. Methodology/Principal Findings: Here we introduce the novel concept of ModuLand, an integrative method family determining overlapping network modules as hills of an influence function-based, centrality-type community landscape, and including several widely used modularization methods as special cases. As various adaptations of the method family, we developed several algorithms, which provide an efficient analysis of weighted and directed networks, and (1) determine pervasively overlapping modules with high resolution; (2) uncover a detailed hierarchical network structure allowing an efficient, zoom-in analysis of large networks; (3) allow the determination of key network nodes and (4) help to predict network dynamics. Conclusions/Significance: The concept opens a wide range of possibilities to develop new approaches and applications including network routing, classification, comparison and prediction.Comment: 25 pages with 6 figures and a Glossary + Supporting Information containing pseudo-codes of all algorithms used, 14 Figures, 5 Tables (with 18 module definitions, 129 different modularization methods, 13 module comparision methods) and 396 references. All algorithms can be downloaded from this web-site: http://www.linkgroup.hu/modules.ph

Emergence of Bursts and Communities in Evolving Weighted Networks

Understanding the patterns of human dynamics and social interaction, and the way they lead to the formation of an organized and functional society are important issues especially for techno-social development. Addressing these issues of social networks has recently become possible through large scale data analysis of e.g. mobile phone call records, which has revealed the existence of modular or community structure with many links between nodes of the same community and relatively few links between nodes of different communities. The weights of links, e.g. the number of calls between two users, and the network topology are found correlated such that intra-community links are stronger compared to the weak inter-community links. This is known as Granovetter's "The strength of weak ties" hypothesis. In addition to this inhomogeneous community structure, the temporal patterns of human dynamics turn out to be inhomogeneous or bursty, characterized by the heavy tailed distribution of inter-event time between two consecutive events. In this paper, we study how the community structure and the bursty dynamics emerge together in an evolving weighted network model. The principal mechanisms behind these patterns are social interaction by cyclic closure, i.e. links to friends of friends and the focal closure, i.e. links to individuals sharing similar attributes or interests, and human dynamics by task handling process. These three mechanisms have been implemented as a network model with local attachment, global attachment, and priority-based queuing processes. By comprehensive numerical simulations we show that the interplay of these mechanisms leads to the emergence of heavy tailed inter-event time distribution and the evolution of Granovetter-type community structure. Moreover, the numerical results are found to be in qualitative agreement with empirical results from mobile phone call dataset.Comment: 9 pages, 6 figure

Northern Eurasia Future Initiative (NEFI): facing the challenges and pathways of global change in the 21st century

During the past several decades, the Earth system has changed significantly, especially across Northern Eurasia. Changes in the socio-economic conditions of the larger countries in the region have also resulted in a variety of regional environmental changes that can have global consequences. The Northern Eurasia Future Initiative (NEFI) has been designed as an essential continuation of the Northern Eurasia Earth Science Partnership Initiative (NEESPI), which was launched in 2004. NEESPI sought to elucidate all aspects of ongoing environmental change, to inform societies and, thus, to better prepare societies for future developments. A key principle of NEFI is that these developments must now be secured through science-based strategies co-designed with regional decision makers to lead their societies to prosperity in the face of environmental and institutional challenges. NEESPI scientific research, data, and models have created a solid knowledge base to support the NEFI program. This paper presents the NEFI research vision consensus based on that knowledge. It provides the reader with samples of recent accomplishments in regional studies and formulates new NEFI science questions. To address these questions, nine research foci are identified and their selections are briefly justified. These foci include: warming of the Arctic; changing frequency, pattern, and intensity of extreme and inclement environmental conditions; retreat of the cryosphere; changes in terrestrial water cycles; changes in the biosphere; pressures on land-use; changes in infrastructure; societal actions in response to environmental change; and quantification of Northern Eurasia's role in the global Earth system. Powerful feedbacks between the Earth and human systems in Northern Eurasia (e.g., mega-fires, droughts, depletion of the cryosphere essential for water supply, retreat of sea ice) result from past and current human activities (e.g., large scale water withdrawals, land use and governance change) and potentially restrict or provide new opportunities for future human activities. Therefore, we propose that Integrated Assessment Models are needed as the final stage of global change assessment. The overarching goal of this NEFI modeling effort will enable evaluation of economic decisions in response to changing environmental conditions and justification of mitigation and adaptation efforts